The present invention relates to the field of psychological evaluation and more particularly, to an improved method of recording the process by which an examinee performs a xe2x80x9cmulti-elementxe2x80x9d test.
It is often necessary to evaluate cognitive abilities of a person, such as memory, organization ability and intellectual capacity, or to identify cognitive skills that have been impaired, for example, by head injuries, neurological disorders, learning disabilities or psychiatric illnesses. To this end, various psychological tests have been developed.
In one common type of psychological tests, an examinee is asked to perform a task involving the manipulation of a finite plurality of elements. Such tasks include copying a complex geometric figure or design. Also included are tasks that include physical manipulation (relocating, assembling, reassembling) of objects with a finite number of elements, e.g. jigsaw puzzles. Also, there exist tests where the examinee is asked to reproduce, perform, recite or recall from memory a finite list of words or commands read by an examiner. For the purposes of succinctness, hereinfurther such tests shall be referred to as xe2x80x9cmulti-element testsxe2x80x9d.
In the art, evaluation of examine performance on multi-element tests is most often based on evaluating the end result of the test, that is, how accurately the examine was able to accomplish the entire test. However, it is known that the strategy that examinees use to accomplish a complex task is indicative of cognitive abilities and deficits, see references 1 and 2. For example, the order and rate by which children copy the elements of a complex figure has been shown to reflect planning ability and visual perception.
Methods have been developed for recording the process by which an examine performs a multi-element test, allowing subsequent evaluation of the process itself. The recording methods described in the prior art include a variety of techniques for manually recording an observed process using xe2x80x9cpencil and paperxe2x80x9d methods
Direct manual recording by an examiner of the sequence of copying elements in a figure-copying test has been described in references 3, 4, 5 and 6.
In reference 6 is described a method whereby an examiner uses a plurality of colored pencils for recording the sequence of manipulation of figure elements by an examine.
References 5 and 6 describe a method whereby the process by which figure-copying tests are performed is recorded by the use of a plurality of colored pencils. Each time the examine copies an important element of a geometric pattern, the examiner gives the examine a different colored pencil. The order in which the important elements are copied is recorded for evaluation. The question as to which element is important is not specified. Moreover, as is clear to one skilled in the art, such a method interrupts the examine and reduces the validity of the results of such a test.
References 5 and 6 also describe a method, by which an examiner mirrors the process of copying a multi-element figure by an examine by copying what the examine copies simultaneously with the examine, and numbering the elements as they are drawn.
Since the pace of multi-element tests is fast and since manual notation is relatively time consuming, the recording methods described in the art fail to record all of the important information concerning the process by which a test is performed. Moreover, since the recording methods known in the art often fail to record a complete description of the process by which an examinee performs a test, it is not clear which part of the process should be given recording priority and how incomplete records are to be evaluated.
For example, in the art there exist more than eight different methods for evaluating the process of copying the Rey-Osterrieth Complex Figure, see references 3, 4, 5, 6, 7, 8, 9, 10 and 11. As known to psychologists, these evaluation methods are complicated and require considerable effort to master. Even when mastered, evaluation of the recorded results requires considerable time and skill. Once the data has been recorded, it can take in the range of 15 to 40 minutes to qualitatively score the test results, see reference 12. Consequently, the process by which an examinee performs a multi-element psychological test is rarely evaluated in a clinical setting, such as hospitals.
The above-mentioned inherent shortcomings of the prior art recording methods make it very difficult to use statistical methods for correlating performance of tests with cognitive abilities. Modem psychological testing theory stresses the advantages of using norms produced by quantitative and objective (i.e. statistical) methods rather than qualitative evaluation of the performance of a large group of examinees, see reference 13. In reference 14, the above shortcomings have been discussed in detail. Despite the increasing recognition of the need to analyze the process by which examinees perform multi-element tests, no method to prepare detailed, reliable and statistically analyzable records of such processes has been described due to the to the shortcomings of the prior art recordings methods.
Methods employing computer-based and multi-media methods for increasing the utility of psychological testing have been suggested. These methods relate to the scoring and evaluation of a test but do not provide a methodology for recording the observed process by which an examinee performs a multi-element task in an ecologically valid manner (reference 15), see for example, U.S. Pat. No. 5,961,332, U.S. Pat. No. 5,991,565 and U.S. Pat. No. 6,115,683.
Other methods have been suggested whereby an examinee actually performs a psychological test using a graphic interface (e.g. monitor) of a computer-based system, for example U.S. Pat. No. 5,211,564, U.S. Pat. No. 5,218,535, U.S. Pat. No. 5,326,270, U.S. Pat. No. 5,379,213, U.S. Pat No. 5,565,316 and U.S. Pat. No. 6,030,226. Although such testing methods have considerable merit for testing the general population, they are not adequate for psychological tests of cognitively impaired, computer illiterate or technologically illiterate populations. Indeed, in reference 15 it has been shown that tasks performed on a graphic interface of a computer-based system lack ecological validity, and do not reproduce the results obtained in a naturally performed task. Thus, computerized tests are qualitatively different and require preparation of dedicated sets of norms.
There is a need for a fast and efficient method for recording, in a complete and reliable manner, the process of performing a multi-element psychological test.
1. Goldstein, K. xe2x80x9cAfter effects of brain injury in warxe2x80x9d. (1942) New York: Grune and Stratton.
2. Werner, H. xe2x80x9cProcess and achievement: A basic problem of education and developmental psychologyxe2x80x9d Harvard Educational Review, 1937, 7, 353-368.
3. Binder, L. M. xe2x80x9cConstructional strategies on complex figure drawings after unilateral brain damagexe2x80x9d Journal of Clinical Neuropsychology, 1982, 4, 51-88.
4. Waber, D. P.; Holmes, J. M. xe2x80x9cAssessing children""s memory productions of the Rey-Osterrieth Complex Figurexe2x80x9d Journal of Clinical and Experimental Neuropsychology, 1986, 8, 563-580.
5. Osterrieth, P. A. xe2x80x9cLe test de copie d""une figure complexe: contribution a l""etude de la perception et de la memoirexe2x80x9d Archives de Psychologie, 1944, 30, 206-356.
6. Rey, A. xe2x80x9cL""examen psychologique dans les cas d""encephalopathie traumatiquexe2x80x9d Archives de Psychologie, 1941, 28, 286-340.
7. Visser, R. S. H. Manual of Complex Figure Test, (1973) Netherlands: Swets and Zeitlinger.
8. Bennet-Levy, J. xe2x80x9cDeterminants of performance on the Rey-Osterrieth Complex Figure Test: An analysis and a new technique for single case assessmentxe2x80x9d British Journal of Clinical Psychology, 1984, 23, 109-119.
9. Hamby, S. L.; Wilkins, J. W.; Barry, N. S. xe2x80x9cOrganizational quality on the Rey-Osterrieth and Taylor Complex Figure Tests: A new scoring systemxe2x80x9d Psychological Assessment, 1993, 5, 27-33.
10. Stem, R. A.; Singer E. A.; Duke, L. M.; Singer, N. G.; Morey, C. E.; Daughtrey, E. W.; Kaplan, E. xe2x80x9cThe Boston Qualitative Scoring System for the Rey-Osterrieth Complex Figure: Description and interrater reliabilityxe2x80x9d Clinical Neuropsychology, 1995, 8, 309-322.
11. Bylsma, F. W.; Carison, M. C.; Schretlen, D.; Zonderman, A.; Resnick, S. xe2x80x9cRey-Osterrieth Complex Figure Test (CFT) Q-scores performance in 328 healthy adults ages 20 to 94xe2x80x9d Journal of the International Neuropsychological Society, 1997, 3, 70.
12. Troyer, A. K.; Wishart, H. A. xe2x80x9cA comparison of qualitative scoring systems for the Rey-Osterrieth Complex Figure Testxe2x80x9d, Clinical Neuropsychologist, 1997, 11, 381-390.
13. Pope, H. S. Butcher, J. N. and Seelen, J. The MMPI, MMPI-2, MMPI-A in Court: A practical guide for expert witness and attorneys, 2d Ed. (2000) Washington, D.C.: American Psychological Press.
14. Poreh, A. xe2x80x9cThe Quantified Process Approachxe2x80x94An emerging approach to neuropsychological assessmentxe2x80x9d The Clinical Neuropsychologist, 2000, 14, 1-11.
15. Sbordone, R. J. and Long, C. J. Ecological Validity of Neuropsychological Testing (1996) Delray Beach, Fla.: GR Press/St. Lucie Press.
16. Meyers, J. E.; Meyers, K. R. xe2x80x9cRey Complex figure under four different administration proceduresxe2x80x9d The Clinical Neuropsychologist, 1995, 9, 63-67.
The above and other objectives are achieved by the innovative recording method provided by the present invention.
The method of the present invention is based on the use of a graphic input device connected to a data storage device used by an observer for recording the observed performance of examinees on psychological tests. The method of the present invention is specifically formulated to be useful in recording the process by which an examinee performs a multi-element psychological test. The method can easily be configured to allow the observer to unobtrusively record the process.
According to the method of the present invention, the observer is presented, through the graphic input device, with a graphic representation of each one of the individual elements of a multi-element psychological test. The graphic input device is configured so that the graphic representation of each individual element is independently registerable by the observer using the graphic input device. When the observer sees the examinee perfonning an element of the test, the observer registers this event using the graphic input device. This registration is stored on a data storage device associated with the graphic input device in a way so that the order in which each element was perfonned and consequently registered is retrievable.
Hereinfurther, the term xe2x80x9cmulti-element psychological testxe2x80x9d is understood to include, but is not limited to, amongst others: the performance of tasks involving the manipulation of a finite number of predetermined elements; the copying of complex geometric figures or designs; the physical manipulation (relocating, assembling, reassembling) of objects with a finite number of elements, e.g. jigsaw puzzles; and the reproduction, performance, recitation or recall from memory of a finite list of words or commands, read by an examiner to an examinee.
Herein, the word xe2x80x9celementxe2x80x9d is understood to be a generic term whose exact meaning is determined by the nature of the specific test being discussed. For example, element may refer to a line to be copied, a word to be recited, or a task to be performed. An element can be physical objects (e.g., blocks or puzzle pieces), graphic shapes (e.g., parts of a drawing), tasks, spoken words or commands. Thus an element is a specific, identifiable portion of a multi-element psychological test.
As used herein, the term xe2x80x9cgraphic interface devicexe2x80x9d means a device configured to (amongst other functions): a) display a graphic representation of an element of a multi-element test; b) allow a user to register each element individually with the help of a corresponding graphic representation; and c) supply, directly or indirectly, the fact of registration and the identity of an individual element registered to the data storage device.
As used herein, the term xe2x80x9cgraphic representationxe2x80x9d means a written or pictorial representation of an element. As is clear to one skilled in the art, it is preferable that a given graphic representation be evocative of an element it represents and that the graphic representation be distinct from the graphic representations of other elements. It is clear to one skilled in the art that most preferable is that graphic representations be non-literal. Non-literal representations allow a single embodiment of the present invention to be used by a plurality of observers who do not share the same language. Further, as is well known to one skilled in the art, identification of a distinct non-literal (that is, pictorial) graphic representation is quicker then identification of a literal graphic representation.
A graphic input device suitable in implementing the method of the present invention includes devices such as a xe2x80x9ctouch-sensitive screenxe2x80x9d, a stylus-activated screen, and a graphic computer monitor coupled with a screen-pointing device (such as a computer mouse).
As stated hereinabove, in order to implement the present invention the graphic input device used is functionally associated with a data storage device. Any suitable data storage device that can be configured to communicate with a specific graphic input device can be used to implement the method of the present invention. However, according to a feature of the present invention, it is most advantageous to use a computer-based device such as a desktop computer, portable computer or a personal digital assistant (PDA). This allows, apart from simple manipulation and storage of collected data, immediate analysis and scoring of a given test. Apart from many obvious advantages, using a computer-based device also allows for the easy recording of the exact time when a specific event is performed in addition to the sequence.
It is not necessary that there be a physical connection between a graphic input device and a data storage device or between a data storage device and a device performing analysis and scoring according to the method of the present invention. It can be countenanced that an observer uses, for example, a PDA with a stylus-activated touch-sensitive screen to record the test-performing process and temporarily store these results. After the test is completed, the observer transmits the gathered results through wireless means to a remote analysis device that evaluates and scores the test, sending back the results of the evaluation to the PDA.
The fact that the process of the examinee performing the test is automatically recorded allows replaying of the process when desired. For example, if the psychological test is performed for use in a judicial process, replaying a recording of the process by which an examinee performed a test overcomes claims of lack of objectivity in interpretation of the results of a test.
Observation of an examinee by an observer includes direct real-time observation (e.g. the observer is in physical proximity of the examinee), remote real-time observation (e.g. the observer is not in physical proximity of the examinee and, concurrently with test performance, observes the examinee through a video device) and non-real time observation (e.g. the observer views a video recording of the examinee performing the test).
The method of the present invention allows an observer to record, accurately and completely, the process by which an examinee performs a multi-element psychological test in a format that allows quick, easy and reproducible analysis. An observer uses a device that includes a graphic input device to record the process of performing the test by the examinee. The graphic input device is coupled to a recording device which records, as a result of the use of the graphic input device by the observer, the order in which and the time when any given part of the test is performed.
In summary there is provided according to the teachings of the present invention a method for recording the process by which an exarninee performs a psychological test, wherein the psychological test includes the stepwise manipulation of a finite number of distinct elements, by:
a) using a graphic input device to display a graphic representation of each one of the finite number of elements of a multi-element test where associated with the graphic representation of an element a corresponding registerable area is delineated and configuring the graphic input device to register indication of a registerable area as selection of the element corresponding thereto;
b) upon seeing a manipulation of a specific element by the examinee, an observer indicating the registerable area associated with the specific manipulated element; and
c) recording the observer""s indication of the registerable area on a data storage device operatively associated with the graphic input device.
It is important to clarify what is meant by association of the graphic representation and the registerable area, as stated immediately hereinabove. Most convenient is that the entire graphic representation of an element be delineated as a registerable area corresponding to that element. This is not necessary and depending on the implementation, a graphic representation can be larger or smaller than a corresponding registerable area, the shape of the graphic representation can be identical, similar or entirely different to that of the corresponding registerable area, and the area covered by the graphic representation can include, partially completely or not at all the corresponding registerable area. Only important is that an observer can, by seeing the graphic representation, quickly identify the corresponding registerable area in order to indicate the registerable area when so desired.
According to the teachings of the present invention, the method of the present invention there is provided a computer device to facilitate communication between the graphic input device and the data storage device.
Still further, according to the teachings of the present invention, there is provided a series of software commands, the software commands configured to, amongst other functions, to facilitate the display of the graphic representations and the use of the registerable delineated areas as described herein.